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Abstract 

Audio feedback has been shown to be popular and well received by students. However, there is little 
published work to indicate how effective audio feedback is in improving student performance. 

Sixty students from a first year science degree agreed to take part in the study; thirty were randomly 
assigned to receive written feedback on coursework, thirty students received their feedback via 
audio files. Mean marks awarded for the coursework for each group were not significantly different. 
The end of module test included questions that specifically assessed topics from the coursework. 
Overall test results were not significantly different for the two groups, nor were marks for the 
coursework-specific questions. 

Samples of the tutor feedback were analysed and the language categorised. The mean word counts 
for audio feedback were significantly higher than word counts for written feedback. Analysis of the 
language used in feedback (measured by word count) indicated significantly higher word counts for 
audio feedback in the following categories, 'explaining misunderstandings' and 'demonstration of 
good practice'. Since word counts for audio feedback might be expected to include a number of 
'filler' words, the number of comments made under different categories was also compared for 
audio and written feedback. Significantly more comments were made using audio feedback in the 
categories 'giving praise', 'explaining misunderstandings', 'demonstration of good practice' and 
'justifying marks'. Under the heading 'suggesting approaches to future work' more comments were 
made using written feedback than audio, although the mean number of comments in both forms of 
feedback in this category was very low. Whilst marks may not be improved for those students 
receiving audio rather than written feedback, the feedback given is much richer. 
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Introduction 

Providing students with timely feedback is a vital part of their learning experience (Gibbs and 
Simpson, 2004; Hattie and Timperley, 2007), but despite the time and effort put into providing 
feedback, this remains an area of Higher Education in which students frequently express 
dissatisfaction (Sadler, 2010; HEFCE, 2012). Part of the problem may lie in the ambiguity of the 
meaning of feedback and the 'assessment literacy' of students (Price et at., 2010) so that whilst 
tutors think they are giving extensive feedback, their students either do not use it and/or do not 
understand it (MacLellan, 2001; Gibbs and Simpson, 2004). In its simplest form, feedback involves 
identification of errors and misunderstandings, but in Higher Education, feedback should be 
addressing much more than this, such as supporting improvements and future development (Gibbs 
and Simpson, 2004; Ferguson, 2011). Current understanding of feedback places more emphasis on 
feed-forward (Price et at., 2010) and communication between tutor and student (Nicol, 2010; 
Dowden et at., 2011). 
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Assessing the effectiveness of feedback is fraught with difficulty. As Hattie and Timperley (2007) 
identify, the timing and type of feedback is important in determining the effect. In some cases 
feedback has been found to decrease performance (Elder and Brooks, 2008). These authors found 
that more elaborate feedback on exam questions (given to students on a nursing science course) 
made no more difference to final scores than simple feedback which just indicated correct/incorrect 
answers. In some cases, exam questions were answered better in subjects in which students had 
been given no feedback during the course. The language used in giving feedback is important in 
determining how students perceive its quality and effectiveness. Weaver (2006) identified feedback 
which was considered unhelpful to students as falling into four areas; 'comments too general', 
'lacked guidance', 'focused on the negative' or 'comments unrelated to assessment criteria'. An 
investigation into students' perceptions of written feedback, (Lizzio and Wilson, 2008) found that 
students valued as most effective, feedback which the researchers termed 'developmental', i.e. 
feedback which guided direction for the student and which could be used in future assignments. 
More recent work identifies the importance of the emotional response of the student to their 
written feedback, so that comments written by a tutor might be perceived as negative (and 
therefore be less effective) although not intended as such (Dowden et al., 2011). 

Traditionally (as in Lizzio and Wilson's work) feedback in Higher Education is given by written 
comments, either handwritten or typed, but audio feedback, (in the form of audio files) has been 
shown to be popular and well received by students, as well as by staff (Lunt and Curran, 2010; Gould 
and Day, 2012). The 'richness' of feedback given can be greater in audio than written feedback 
(Merry and Orsmond, 2008; Gleaves and Walker, 2013). Merry and Orsmond (2008) classified 
language used by tutors in their feedback (e.g. 'identifying errors',' explaining and justifying marks') 
using a coding system produced by Brown etal. (2003) as part of the FAST project (Formative 
Assessment in Science Teaching). Gould and Day (2012) piloted the use of audio feedback with a 
cohort of community nursing students and found that they valued the more detailed, supportive and 
personalised aspects of audio feedback compared to written feedback. Whilst there is evidence that 
both students and tutors might like audio feedback (Chiang, 2010; Gould and Day, 2012), there is 
little published work comparing the outcome of audio feedback in improving student performance. 
Gleaves and Walker (2013) assessed improvements in essay writing amongst students who either 
received written or aural feedback, but found that the format of feedback had no significant effect 
on the outcome. Likewise, Macgregor et al. (2011) found that, despite high levels of re-use by the 
recipients, a group of 24 students who were given extensive formative feedback by means of 
'voicemails', showed no significant learning gains when compared with the control group. 

The study described in this paper was undertaken to address the question in relation to writing a 
laboratory report. By providing feedback on a piece of coursework in either audio format, or in the 
more traditional format of written comments, to separate groups of students, and then assessing 
these students on topics raised by the coursework, we aimed to identify if learning was significantly 
improved by the format of feedback given. In addition, following the work by Merry and Orsmond 
(2008), the language used in audio versus written feedback was analysed according to the categories 
given by Brown et al. (2003). 

Method 

A cohort of 120 students studying on a practical module for a degree in Biological Sciences in the 
Faculty of Health, Life and Social Sciences at Edinburgh Napier University, were approached and 
asked for informed consent to take part in the study. The five members of staff involved in teaching 
this module were also asked. The Faculty Research Ethics Committee gave approval for the study. 
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Sixty students signed up for the project as did all five members of staff. The coursework consisted of 
sections of a laboratory report (introduction, materials and methods, results and discussion), which 
were then marked using written feedback (hand written or typed) or audio feedback. The students 
were randomly assigned (30 to each group) to receive either written or audio feedback on their work 
from all five members of staff. The staff were provided with a proforma for giving written feedback, 
but they were free to give typed or handwritten comments in addition to the given categories. They 
were also provided with portable digital recorders (Sony ICD-UX71) to record mp3 files. In giving 
audio feedback, they were free to do this as they wished, but were given training in using the 
recorders, and some guidelines e.g. regarding the length of recording (Rotheram, 2009). Audio files 
were then emailed to students as mp3 files. Staff were informed of the aims and that transcripts 
would be made of their audio files, but were not told that an analysis of their language would be 
undertaken since it was felt that this would influence their style in marking. Transcripts of a sample 
of audio files and samples of written feedback from each tutor were kept for analysis. Comments 
from the transcripts and written feedback were then categorised making use of the coding system 
devised by Brown et al. (2003) and by Merry and Orsmond (2008). One category used by Merry and 
Orsmond ('engaging students in thinking') was not used since the length of the assignment (1000 
words), and its nature (first year science laboratory reports), meant that comments by markers were 
better categorised under the other headings. 

Mean word counts, and mean number of comments for each category (across all markers) were 
calculated from the samples, and compared between written and audio feedback. Comparison of 
means was undertaken using unpaired, two tailed student's t-test. A p value of <0.05 was taken to 
be significant. 

At the end of the module, all students undertook an on-line test containing questions about writing 
a laboratory report. Results from the students who took part in the project were analysed to see if 
the form of the feedback affected the outcome in the test. In addition, in their next semester, 
students were assessed by means of a full laboratory report, marks for this were analysed according 
to the two groups. 

Once feedback had been returned, students and staff were asked to complete a questionnaire 
seeking information about their experience of either giving, or receiving feedback in different 
formats. 

Results 

Marks awarded for coursework and on-line test results 

The mean marks awarded for the coursework, and for the on-line test questions, for students in the 
different feedback categories, are shown in Figure 1. 



Audio feedback group 
n= 30 

Written feedback group 
n= 30 

Student's 

t-test 
p value 

Mean mark awarded for 
coursework 

53.1% 

51.1% 

0.6 

Mean mark for on-line test 
questions relating to coursework 

74.7% 

72.6% 

0.76 

Mean mark for lab report in 
following semester 

54.4% 

53.8% 

0.86 


Figure 1 . Mean marks awarded for students in the two feedback groups. 
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The remaining cohort of students who did not take part in the study, achieved a mean mark of 46.1% 
for the coursework (non submission excluded) which was significantly lower than the mean for the 
audio group (p = 0.04), but not significantly different from the written feedback group (p = 0.2). For 
the on-line test, the non-participating cohort achieved 72.4% in the relevant questions. This is very 
similar to the results from the participating groups (see Figure 1.). 

Analysis of language 

A comparison of total word counts from the sampled transcripts of audio files and written feedback 
(from all staff) revealed a significant difference between the two, 482.2 +/- 344.1 words for audio 
feedback versus 93.9 +/- 88.0 for written feedback (student's t test, two tailed, unpaired p = 0.04). 
The large standard deviation for word counts was mostly attributed to one member of staff who 
produced audio reports of almost 900 words and written reports which were more than twice as 
long as those from other staff. 

Transcripts of samples of the audio files, and copies of samples of the written feedback were 
analysed. The numbers of words used by tutors, were categorised under the following headings, as 
were the number of comments in each category (after Brown et al. 2003), see Figure 2. 


CATEGORY 

EXAMPLE OF PHRASE USED IN AUDIO OR WRITTEN 

FEEDBACK 

Identification of errors 

'No explanation of the analysis of raw results' 

'I'm not sure why you separated this into three headings?' 

Giving praise 

'good' 'OK' '...was done well' 

'your taxonomic graph was good' 

'You've got a good heading' 

Correcting errors 

'You need to be guoting numbers in those results' 

'never number references' 

Explaining misunderstandings 

'here you are . you need to ...instead' 

'The reason it's ., but 1 would have thought . So if you 

ever see that in the future....' 

Demonstration of good practice 

'you then go on to...which was absolutely fine and how it 
works.' 

'you also found a reference for . so 1 was really 

impressed.' 

Suggestions for future study 

'refer to tutorial two, that shows you a better way of 
doing it.' 

"take care with careless mistakes like this" 

Justifying marks 

'You haven't lost marks for this, but it's good to do that in 
the future.' 

'...you would have been able to achieve good marks if you 
had done that' 

Suggesting approaches to future work 

'Please, in future, can you use Aria! or Calibri 12 font.' 


Figure 2. Examples of phrases used in feedback under different categories 
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Comparison of the total word counts in the categories of feedback for the audio versus written 
feedback groups is shown in Figure 3. 



Figure 3. Comparison of the types of comments used in samples of audio versus written feedback. 
The bar graph shows mean word counts in each category with error bars indicating standard errors 
of the mean. 

There were significant differences between the means indicated by * (p=0.025) for 'Explaining 
misunderstandings' and ** (p=0.004) 'Demonstration of good practice' on Figure 2. In the category 
'Justifying marks' the difference between word counts for audio feedback and written feedback was 
not quite significant (p=0.052). 

It could be expected that word counts for audio feedback would be greater than written feedback 
since audio feedback would include 'filler' words and instructions to students about which section 
the reader was referring to. As well as counting the total number of words used in feedback under 
each heading, the number of comments made was analysed. 

Figure 4. compares the number of comments made using written versus audio feedback. 
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Figure 4. Comparison of the types of comments used in samples of audio versus written feedback. 
The bar graph shows mean number of comments in each category with error bars indicating 
standard errors of the mean. 

A comparison of the number of comments made showed significantly more comments made using 
audio feedback, indicated by the asterisks, under the headings 'giving praise' (p=0.006), 'explaining 
misunderstandings' (p=0.0025), 'demonstration of good practice' (p< 0.0001), and 'justifying marks' 
(p< 0.0001). The only category in which more comments were made using audio feedback than 
written, was under the heading 'suggesting approaches to future work' for which written feedback 
gave significantly more comments than audio (p=0.008), although the mean number of comments in 
this category was very low. 

Student evaluation 

Responses were received from 21 students who received written feedback and 16 who received 
audio feedback. Students had broadly similar opinions on a number of issues, for example a 
majority in each group felt that their feedback was a mix of positive and negative comments. A 
majority in each group also agreed that the feedback gave suggestions as to how they could improve 
in the future. However, only 52% of the written feedback group, and 50% of the audio group felt 
that the feedback justified the marks awarded. 


81% of the audio feedback group agreed that their feedback was clear and easy to follow, compared 
to 47% in the written group. Comments offered by a student in the 'written feedback' group give 
one reason for this: 
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'writing should be readable; I have trouble reading peoples writing so I would prefer to have 
my feedback audio.' 

There were positive comments about both types of feedback: 

'referred audio feedback, more detailed, gave more information, explained better where I had 
gone wrong and what I needed to improve on' (audio). 

7 iiked how it went through step by step and explained everything' (written). 

Staff evaluation 

Comments from the four members of staff who responded to the evaluation, indicated mixed views 
about using audio feedback. One member stated that they would like to continue using audio 
feedback, but that it depended on the assessment type and on the number of scripts. Another felt 
that the benefit of giving written feedback was that they were able to give an example of how e.g. a 
graph should look, which wouldn't be possible using audio feedback. 

The time taken to give the feedback was an issue for one participant who found that they had to 
produce written feedback before they could produce an audio file, hence doubling the time taken. 
There was one comment relating to the depth of feedback given with the audio files: 

'the temptation with written feedback is to make it short.' 

Discussion 

In its simplest form, feedback might be seen as a way to improve performance and grade. This study 
did not find a quantifiable difference in the outcome of test results following different formats of 
feedback given, whether written or audio feedback. The authors acknowledge that a selection of 
questions in an on-line test is not a powerful tool for assessing learning, and it would be surprising to 
see a difference made simply due to the format of feedback, nevertheless, we feel this was a 
valuable exercise to undertake. In addition, the follow up marks in semester 2 for the laboratory 
report might be seen as an indication of learning from feedback in semester 1 since the assessment 
types were similar, but again, it would be surprising to have found a significant effect due to the 
format of feedback. This is in line with the work by Macgregor et at. (2011) who found no significant 
effect on learning gains amongst Business Management and Information students who had access to 
either formative audio feedback, or written feedback. Gleaves and Walker (2013) present a similar 
picture from their study which compared formative audio and written feedback, on the outcome of 
essay writing amongst students studying for a BA Education. These authors found no significant 
differences in 'knowledge elaboration' (the process of using prior academic and personal knowledge 
to refine and construct new material) in essay construction, between two student groups, one which 
received written formative feedback, the other audio feedback in the form of audio files. The group 
receiving audio feedback commented on the greater "richness" provided (a finding of many studies 
into audio feedback, Merry and Orsmond, 2008; Lunt and Curran, 2010; Gould and Day, 2012) but 
the researchers identified this as relating to feedback in the form of confirmation and reassurance to 
students, rather than in promoting continual improvement. 

The students who signed up for taking part in this study may well have been students who were 
more interested and engaged with the topic, and this might explain the reason for the significant 
differences between the marks awarded for coursework amongst the 'audio' group (53.1%) and the 
students who did not take part in the study (46.1%), although there was no significant difference 
between those receiving written feedback (51.1%) and the remaining cohort. Students were 
randomly assigned to the different groups, and it might be that the higher mark reflects a tendency 
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for tutors to be more generous when giving audio feedback. A trial of marking students' work "face 
to face" found that staff gave significantly higher marks to these students than to their peers who 
received written feedback (Chalmers et al., 2013). 

The analysis of the language used was undertaken using a similar tool to that described by Merry 
and Orsmond (2008), based on guidelines produced by Brown et al. (2003). Whilst our findings are 
not directly comparable due to differences in methodology, in the category of 'demonstrating good 
practice' our results are in line with Merry and Orsmond (2008) who also found that audio feedback 
gave significantly more (in terms of word count) than written feedback. The amount of feedback 
given by audio files is clearly greater (see mean word counts), but does more feedback mean better 
feedback? Elder and Brooks (2008) investigated the effect of 'simple' or 'elaborate' feedback on 
exam performance amongst graduate nursing students and found that whilst some feedback was 
better than none, the extent of feedback given did not affect performance. Students performed 
worse in a later exam, in subjects on which they had previously been tested and had received 
elaborate feedback, performing better in subjects which had previously been tested and on which 
they had only been given minimal feedback. The authors concluded that the minimal feedback had 
promoted deeper engagement with the topic and therefore a better long term outcome in terms of 
learning. 

Of the 37 students who responded to the questionnaire, only 50% of the audio group felt that the 
feedback justified the marks awarded (52% for the written group), despite the category of feedback 
labelled 'justifying marks' providing more feedback using audio files, than with written feedback. 
Staff in Higher Education spend much time in marking work and providing extensive written 
feedback for students, but there is clearly a mismatch between what staff think is provided (Bailey 
and Garner, 2010; Price et al., 2010), and what the students think they receive, as consistently poor 
student surveys show (HEFC, 2012). Weaver's study (2006) into the students' perceptions of 
feedback indicates the aspects which they do not find helpful as being; vague comments, feedback 
lacking guidance or focused on the negative, and feedback unrelated to assessment criteria. 
Likewise, Ferguson's study (2011) identified that what students (undergraduate and post graduate) 
value most in the feedback they receive are positive, clear and constructive comments which guide 
them to future improvements. Lizzio and Wilson (2008) also investigated student perceptions of 
feedback; students identified the most effective feedback as being 'developmental', i.e. that which 
could help towards development in future work. In our study, feedback categorised under 
'suggesting approaches to future work' which might be deemed as 'developmental', did not feature 
strongly in either audio or written feedback. This was also the case in the study by Merry and 
Orsmond (2008). 

Whilst the study did not specifically set out to explore staff views towards using audio feedback, the 
comments received are in line with those of Chiang (2010) and Gould and Day (2012) in that there 
were very mixed views about using audio feedback and the possible increase in workload for them. 
However, Lunt and Curran (2010) found the tutor experience of using audio feedback to be more 
positive, with two tutors finding that audio feedback took much less time than writing for a 2000 
word coursework. This may reflect different types of work being assessed; the comment by one 
member of staff in our study, regarding the appropriateness of using audio feedback to explain 'how 
a graph should look'[ echoes those of Chiang (2010) who identifies that assessors should match the 
type of feedback not only to the assessment, but also to the learner. This obviously becomes 
important in the case of students with hearing difficulties or for students whose first language is not 
English. 

What does this study tell us about audio feedback and how this relates to good practice? There is 
now plenty of evidence that many of our traditional methods of giving feedback to students are 


71 



CHALMERS, MACCALLUM, MOWAT & FULTON: AUDIO FEEDBACK: RICHER LANGUAGE BUT 

NO MEASURABLE IMPACT ON STUDENT PERFORMANCE 


putting an increasing burden on staff (Bailey and Garner, 2010) and yet are either not working (Price 
et at., 2010), or are not perceived as being helpful (Weaver, 2006; Lizzio and Wilson, 2008). Audio 
feedback has the potential to fulfil some of the criteria for good feedback, namely timeliness (it can 
be quicker to get the feedback to the student); in giving more detail to the student without 
increasing the marking burden (Lunt and Curran, 2010), and in making feedback personal (Gould and 
Day, 2012). Students are also receptive to this type of feedback and are much more likely to listen to 
it than to read written feedback (Lunt and Curran, 2010). Resistance by staff may be reduced with 
the emergence of quick and easy to use software for personal computers, tablets and phones 
making it easier to use audio feedback if they so wish, without feeling an added burden of dealing 
with 'difficult' technology. In this, and other studies, the effectiveness of audio feedback, has not 
shown to be any more effective than using written feedback, (Macgregor eta!., 2011; Gleaves and 
Walker, 2013), and it might be that the type of work being marked will to a large extent determine 
the best format for giving feedback. The principles of being positive, consistent in linking feedback to 
assessment criteria and in focussing on future improvement relate to any form of feedback. 
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